Add optional Implementations section #541
Conversation
act-rules-format/act-rules-format.bs
Outdated
| @@ -544,6 +545,8 @@ Test Cases {#test-cases} | |||
|
|
|||
| Test cases are (snippets of) content that can be used to validate the implementation of ACT Rules. Each rule <em class="rfc2119">must</em> have one or more test cases for `passed`, `failed`, and `inapplicable` [=outcomes=]. A test case consists of two pieces of data, a snippet of each [input aspect](#input-aspects) for a rule, and the expected outcome for that rule. Test cases serve two functions, firstly as example scenarios for readers to understand when the outcome of a rule is `passed`, `failed`, or `inapplicable`. It also serves developers and users of accessibility testing tools and methodologies to validate that a rule is correctly implemented. | |||
|
|
|||
| All `passed` and `inapplicable` test cases of an ACT Rule <em class="rfc2119">must</em> satisfy at least one of that rule's [=accessibility requirements=]. For all `failed` test cases, all accessibility requirements <em class="rfc2119">must</em> be *not satisfied*. | |||
There was a problem hiding this comment.
This is not a backward compatible change. We need to think about how we want to handle that. Maybe only require this if the rule has metadata indicating it was created for ACT-RF 1.1?
act-rules-format/act-rules-format.bs
Outdated
| @@ -544,6 +545,8 @@ Test Cases {#test-cases} | |||
|
|
|||
| Test cases are (snippets of) content that can be used to validate the implementation of ACT Rules. Each rule <em class="rfc2119">must</em> have one or more test cases for `passed`, `failed`, and `inapplicable` [=outcomes=]. A test case consists of two pieces of data, a snippet of each [input aspect](#input-aspects) for a rule, and the expected outcome for that rule. Test cases serve two functions, firstly as example scenarios for readers to understand when the outcome of a rule is `passed`, `failed`, or `inapplicable`. It also serves developers and users of accessibility testing tools and methodologies to validate that a rule is correctly implemented. | |||
|
|
|||
| All `passed` and `inapplicable` test cases of an ACT Rule <em class="rfc2119">must</em> satisfy at least one of that rule's [=accessibility requirements=]. For all `failed` test cases, all accessibility requirements <em class="rfc2119">must</em> be *not satisfied*. | |||
There was a problem hiding this comment.
Should we mention this is not about the secondary requirements?
(i.e. Failed case do not need to fail all secondary requirements)
act-rules-format/act-rules-format.bs
Outdated
| <dt><dfn>Implementation Procedure</dfn><dt> | ||
| <dd> | ||
| <p>A procedure resulting in an [=outcome=]. The procedure can be a step by step description of how to manually perform the test, a fully automated test, or some combination of manual and automated testing.</p> | ||
| </dd> |
There was a problem hiding this comment.
(
So, a (mathematical or computer program) function can either be described as an intention, i.e. a formula or code to compute it (e.g. x => x*x is an intention for the squaring function); or it can be described as an extension, i.e. the set of (input, output) pairs (e.g. { (0, 0), (1, 1), (2, 4), ... }for the squaring function.
Intentions are usually easier to manipulate and we are used to them. Extensions are sometimes useful to consider.
Anyway, here the procedure/check is described as an intention ("a step by step procedure"). But ultimately, the only thing that matters for consistency is the outcomes it gives, i.e. its extension.
So, I'm wondering whether it could make sense to instead define the procedure/check as "a set of outcomes" rather than as the piece of code that produces this set.
It may be a bit harder to grasp at first, but it might allow to simplify the other definitions down the line. It is also a bit closer to what we do in practice since we work with the implementation reports (an extension) rather than the code itself (an intension).
| A [=set of checks=] is [=consistent=] with an ACT Rule if it meets all requirements for a [=consistent|consistent check=]. The [=outcomes=] for this set of checks is a list of all outcomes of checks that are partially consistent with the ACT Rule. The accessibility requirements of this set of checks is a list of all accessibility requirements of checks that are partially consistent with the ACT rule. | ||
|
|
||
| <aside class=example> | ||
| <header>Example of a consistent set of checks:</header> |
There was a problem hiding this comment.
Really like this example!
Co-authored-by: Trevor R. Bostic <[email protected]>
act-rules-format/act-rules-format.bs
Outdated
| @@ -544,6 +545,8 @@ Test Cases {#test-cases} | |||
|
|
|||
| Test cases are (snippets of) content that can be used to validate the implementation of ACT Rules. Each rule <em class="rfc2119">must</em> have one or more test cases for `passed`, `failed`, and `inapplicable` [=outcomes=]. A test case consists of two pieces of data, a snippet of each [input aspect](#input-aspects) for a rule, and the expected outcome for that rule. Test cases serve two functions, firstly as example scenarios for readers to understand when the outcome of a rule is `passed`, `failed`, or `inapplicable`. It also serves developers and users of accessibility testing tools and methodologies to validate that a rule is correctly implemented. | |||
|
|
|||
| Every `passed` and `inapplicable` test cases of an ACT Rule <em class="rfc2119">must</em> satisfy all the rule's [=conformance requirements=]. For every `failed` test cases, all conformance requirements <em class="rfc2119">must</em> be *not satisfied*. | |||
There was a problem hiding this comment.
This tripped a few folks up, could do with more of an explanation.
Co-authored-by: Trevor R. Bostic <[email protected]>
SHA: 27f92b5 Reason: push, by WilcoFiers Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
SHA: 27f92b5 Reason: push, by WilcoFiers Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Follow-up of #522 (which was out of date and had merge issues)